This invention relates to electronic shopping agents or xe2x80x9cbotsxe2x80x9d which operate over the Internet on behalf of a client to locate on-line vendors which provide goods and services of interest to the client.
The Internet and web-related technology have become widespread as personal computers have become more prevalent. One of the fastest growing business sectors is electronic commerce, particularly, retail consumer shopping. The Internet allows consumers to quickly locate goods and services of interest to them. In many cases, images of the goods can be viewed and orders placed directly over the web. The consumer may provide payment electronically via credit cards and the goods are then shipped to the consumer. Comparative shopping using the Internet as a search and retrieval tool to locate and retrieve information and prices for comparable products is also a fast-growing area. There are already several comparative shopping tools that are available on the Internet, such as Junglee and Jango, for example. These shopping tools accept keywords and category information as inputs from consumers. The keywords and category information are used to create an autonomous agent or xe2x80x9cshopping botxe2x80x9d which scans over the Internet and locates related products from a set of online merchants. The product items that are located are returned by the shopping bot are then presented to the consumer using a simple tabular form to enable comparison shopping.
While the existing shopping tools can help users to do comparative shopping, there are several limitations among all the existing shopping bots. The first limitation is that the number of online merchants included in the comparison pool that a user can access and use for comparisons is small. This limitation can be mainly attributed to a historical fact, i.e. the evolution of the Internet. The Internet was originally designed to operate with information coded in a very specific format called HyperText Markup Language (HTML). HTML is a presentation language that uses codes embedded in the document to define how a particular segment of a document is presented on a display mechanism such as a Web browser. Although HTML has a predefined and fixed format, it does not give any information about the meaning or semantics of the information which it is used to format. Therefore, although Web browsers can read HTML and use the HTML codes to identify selected parts of the information, such as text and graphics, the browsers cannot use the HTML codes to extract information from the identified document parts. In addition, web pages often differ drastically depending on the taste, preference, and marketing needs of different designers and merchants.
As a result, it is difficult and tedious for a search engine to extract specific information, such as item price, from a wide variety of different HTML coded web pages. In order to overcome this problem, some shopping bots use mechanisms such as so-called xe2x80x9cwrappersxe2x80x9d or xe2x80x9cinformation adapters.xe2x80x9d One of these mechanisms is programmed to discover or xe2x80x9clearnxe2x80x9d about each product category in each merchant site. However, these mechanisms are very slow and it usually takes from hours to days to include a new merchant in the comparison pool. Furthermore, if the merchant changes its web page formats, it will take also a long time to change the mechanism used for that merchant site in order to make the search engine continue to work. The continual flux of the Internet requires the shopping bot providers to employ many programmers to design and maintain their services and the number of merchant sites covered is necessarily small.
The second limitation is the amount of information a consumer can get through existing shopping bots. Currently, most shopping bots provide consumers with only limited information such as price, a brief description of items, and a merchant link. However, from consumer""s point of view, price may not be the only criteria on which to base a shopping decision. Other factors, such as shipping date, warranty information, creditability of a merchant, and service, etc. often affect shopping decisions. Although it is possible to add additional criteria to existing shopping bots, it means changing hundreds or thousands of xe2x80x9cwrappersxe2x80x9d or xe2x80x9cinformation adaptersxe2x80x9d corresponding to different categories, subcategories, and merchant sites. Such a task is a very costly investment both in terms of time and human resources.
The third limitation on existing shopping bots is performance and accuracy of searched results. Most existing shopping bots are very slow and take minutes to generate search results. Consequently, many existing systems store information retrieved from merchant sites in a local database so that searches are greatly accelerated. However, the local databases are only periodically updated by contacting the merchant sites. Therefore, the results of the search are often out-of-date and not very accurate or not related to the actual situation at the merchant site.
Therefore, there is a need for a shopping bot which can operate with a variety of different merchant site formats and which can quickly adapt to new formats or changes to existing merchant sites.
There is further need for a shopping bot which can be easily and quickly modified to retrieve and display new and different information from that currently being displayed.
There is a further need for a shopping bot which can quickly provide accurate and timely information to consumers.
In accordance with the principles of the invention, a shopping bot uses real time agents that automatically contact disparate web pages representing vast variety of different categories and merchants and retrieve and unify the information therein for display when a request for the information is made. Consequently, there is no need to create a xe2x80x9cwrapperxe2x80x9d, or a xe2x80x9cinformation adapterxe2x80x9d for each category or each merchant because the same agent can retrieve and process information in various formats.
In particular, the shopping bot generates queries from keywords entered by a user and a database of URL information. Information returned by the queries is filtered, parsed and mapped to a standard format. The formatted information can then be displayed. Since the information is converted to the standard format in real time, the invention allows quick addition of online merchants and additional product categories can be added quickly and easily. Further, information about a product item can be easily enlarged as market needs increase without changing a database of codes one-by-one.
In accordance with one embodiment, the database of URL information includes URLs specific to site directories at each merchant site so that queries can be easily generated by appending user-provided keywords.
In accordance with another embodiment, information which is retrieved from merchant sites comprises information which would normally be displayed by a browser. This information which is typically in HTML or XML format is parsed and filtered and a hierarchical tree structure is used to map the information to desired categories before displaying the information for comparison.
In accordance with still another embodiment, efficient caching and distributed algorithms are used to reduce consumer response time.